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ABSTRACT 
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whether it is possible to tutor effectively with incomplete knowledge 
in a complex domain, A 20-item reference list is included. 
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Abstract 

The work described here forms part of a project using models of musical ideas within 
an AI and education framework whose goal is to encourage and facilitate music 
composition by novices. Formal knowledge of the domain (popular music) is too 
incomplete and fragmented to support a traditional expert-based tutor for precisely 
constrained tasks with clear-cut rules. Instead the proposed system will use a net of 
disparate, incomplete knowledge sources linked with a small number of microworids. 
The proposed system includes a new microworld for harmony, based on recent 
cognitive theories. The knowledge-based component will take the form of a net of 
objects corresponding to knowledge about pieces of music, musical styles and the 
musical strategies and techniques employed in them . The net is connected by links of 
generality, specificity and similarity. The proposed knowledge base is intended be 
explored in its own right, used to transform and generate fragments of music and used 
to guide discovery in the microworids. The system will try to construct teaching plans 
on the basis of what a novice can do, and equally importantly, what he likes in order 
to help him find paths to personally important goals. As well as an attempt to tackle 
the stringent demands of the particular domain, the research is an exploration of 
whether it is possible to tutor effectively with incomplete knowledge in a complex 
domain . 
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This is an extended abstract of a paper that was presented at the 3rd International Conference on 
Artificial Intelligence and Education, Pittsburgh, Pennsylvania 9lh, May 1987. This version was 
prepared Nov 86 with minimal revisions made June 87. A more detailed and fully revised version of the 
paper including details of work carried out since Nov 86 is in preparation . 

Introduction 

The work described here forms part of a project whose goal is to find ways of using 
AI models of music composition within an AI and education framework to encourage 
and facilitate music composition by musical amateurs. We describe here design 
considerations for a knowledge-based ICAI system that tries to construct teaching plans 
on the basis of what an amateur can do musically, and equally importantly, what 
music he likes, in order to help him find paths to his chosen musical goals. 
Btcause knowledge by itself is not enough, musical experiments and experiences 
need to be made available. The system has components for the direct manipulation of 
harmonic and rhythmic materials, and for assembling and combining musical devices 
built out of these materials. These are designed to be accessible to non-expert musicians 
without instrumental skills. 



1 Some possible teaching stratepies fo r composition 

Let us imagine someone who is musically uneducated and wishes to compose music 
but doesn't know how to set about it. Equally we can imagine someone with a 
traditional or idiosyncratic music education who can't compose. A human tutor might 
begin by focussing on three questions. 
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a/ What sort of music the amateur wants to compose, 
b/ What sort of music the amateur likes or is interested in. 
c/ What musical skills the amateur presently possesses. 

The teacher might dynamically construct a teaching plan and exercises tailored to that 
amateur's likes and capabilities while making reference to a core curriculum of 
generally useful compositional strategies, techniques and materials. One teaching 
strategy might be to provide intermediate goals based on the amateur's self -expressed 
preferences in music as given in answer to the initial questions. A human teacher might 
say "The student wants to compose like (say) Steely Dan, so he is going to need to 
know about voicing 9th, 11th and 13th chords, about using cycle of fifths 
progressions, about transient modulation, about use of rhythmic motifs, melodic 
trajectories (Levitt, 85), and so forth. He doesn't know about any of this yet. We could 
start with a simple blues, and then look at some simple Jazz standards using cycles of 
fifths...". This approach appears reasonable, provided the student is willing to settle 
for them as enjoyable substitute goals for the time being, or is convinced of the 
relevance of the intemiediate steps to reaching his goal. A conn-a.sting strategy would 
be to say " Composing like Steely Dan is beyond this student's capabilities as yet, but 
by playing with these simple components he could get a very rough approximation 
right now". A reasonable combined stru.fjgy might be to offer the amateur at all points 
a free choice between these two strategie'? (personally meaningful intemiediate steps or 
immediate rough and ready results;. 

2 Domain Knowledge 

The system is to specialise in fairiy limited areas of popular music and Jazz, but needs 
a base of knowledge common to most Western tonal music. (In principle it could be 
extended to deal with other kinds of music wherever appropriate music analysts and 
teachers could be found.) Several quite different layers of domain knowledge are found 
to be needed.Three layers of tools are proposed to help learn about them. The bottom- 
most layer of tools consists of experiential tools that can be used in different ways: 
freely; with fixed-conicnt worksheets; or under guidance from the topmost knowledge- 
based layer. 

2.1 Primitive materials layer 

1/ A direct manipulation environment ("2D harmony space") for experimenting with 
harmonic relationships will be provided as discussed in (Holland 86) . The amateur can 
analyse, amend and sketch chord sequences and should be able to understand and 
control chord progression, chord quality and modulation from simple experiments. The 
environment can be used on its own; using simple fixed exercises without tutorial 
guidance; or the ICAI component can construct exercises for the amateur to carry out in 
the environment with the system inspecting results. 
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2/ A direct manipulation environment for specifying and exploring rhythmic patterns 
based on the notion of hierarchical metre as characterised by Lerdahl and Jackendoff 
(1983) using a simple visual metaphor will be provided. (The term "metre" refers to 
layers of regular pulses found in music at levels corresponding to, for example, eighth 
notes, quarter notes, half-bar lines, bar lines, two bar lines etc.) The tool allows 
rhythmic motifs or other musical events to be "scaled" and propagated at any metrical 
level. Such operations can be easily used to set up patterns of rhythmic accompaniment 
or simple repetitive musical forms. This component can be viewed as a musical editor 
in which hierarchical metre is used with a simple visual metaphor to delimit scope for 
editing, scaling and propagation operations. Work by Jean Bamberger (1974) using a 
single-level representation of metre suggests ti.at such an environment should be a 
good source of insights into how metre interacts with phrasing, and that such insights 
play a vital part in understanding musical structure. This component does not 
necessarily include tutorial guidance, but once again, the ICAI component can constiuct 
exercises to be carried out in this component. Note that this tool, though useful, does 
not begin to address the full subtleties of rhythm. 

3/ (For completeness, a component is needed here for the direct manipulation of 
different melodic strategies, but time limitations are at present restricting work on this 
component. It appears that a tool could be designed using the key musical idea of 
trajectory (i.e scalic movement towards a target note) (Levitt, 85) , and the knowledge 
representation method of propagation of symbolic constraints (i.e. a kind of musical 
spreadsheet). Much of theoretical work for such a system has been done by (Levitt, 85) 
but the design of good human machine interfaces for controlling such systems, 
especially for use by novices, remains an open research problem.) 

2.2 Musical ideas direct manipulation layer. 

This layer is conceptually the simplest of all, although its content might get very 
complex - Music Logo (see (Bamberger, 86), (Orlarey 86), (Desain and Honing, 86) 
and (Greenberg 86)). In a Music Logo, trees of functions can be assembled to model 
musical ideas of varying degrees of complexity. It is intended ultimately to provide a 
simple direct manipulation iconic interface in which boxes (representing functions) with 
funnels (representing arguments and outputs) ivrt bolted together. This does not imply a 
data-flow model of time; time will be represented as line intervals given as input 
arguments. Both the basic materials layers below and the ICAI layer above could be 
used as sources of input to this layer - musical ideas assembled in the Music Logo 
component could become part of the linked frame system used by the ICAI layer. The 
Logo is to be available for unguidcd use, and as with the other lower layers, the ICAI 
component could construct exercises to be carried out in the Ixjgo layer. 

2.3 Knowledge-based layer 

The topmost layer, the tutor, requires a linked set of active objects or frames (the two 
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terms are used synonymously here) representing knowledge about a wide range of 
musical items. Kinds of frames needed (with much overlapping) include:- 

1/ Primitive pitch, rhythm and harmonic materials. 

21 Primitive devices and techniques used in composition such as melodic trajectory, 
harmonic progression in cycle of fifths and phrasing functions of rhythm. 

3/ Heuristics and tricks (of the sort found in "how to" music books) for combining 
primitive material and devices in interesting ways. 

4/ Broad frameworlcs in which to construct pieces (e.g. thread a chord sequence with 
a melody, have voices imitate each other, stack functional layers, stack ostinati 
(Reck,77)). 

5/ Different ways to construct a piece in terms of a goal or script (e.g. begin 
predictably, thwart prediction, make it retrospectively clear why prediction was 
thwarted (adaption of (Levitt, 81)). 

6/ Resyntheslsed versions of fragments or aspects of complete pieces, broken 
down as far as possible into melodic, harmonic and rhythmic materials, devices and 
strategies. 

7/ Partial characterisations of styles represented as in the previous item (Jones, 
86) (Bamberger,86) etc). It should be noted that the models to be used here are aimed 
at engaging amateurs' interest, not necessarily at scholarly correctness. 

Objects are linked by partial order relations of specificity and generality and by a 
similarity net. Attached procedures can carry out tasks such as compose fragments in a 
given style or construct teaching exercises using specified material when asked. As 
well as carrying out suggested exercises, the student can browse and fill in slots or 
create new frames under his own direction. The activity of filling slots corresponds to 
the student composing new pieces, fragments, strategies or devices and finding links 
between existing objects. In addition to pre-existing links , the student's .subjective 
judgement of where similarity and specificity and generality links should be forms part 
of the student model and should be recorded separately. Example frames will be 
shown in th^* '^^esentation and their interaction in an imaginary scenario will be 
examined. 



2. 4 User model 

A user model is held, consisting of an agenda of the user's stated and inferred goals, 
and two kinds of overiay. Both overiays are of musical material represented both as 
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wholes and as components. One overlay represents the student's skills, and another 
music he enjoys. As outlined above, the latter is used to help construct enjoyable 
intermediate goals and exercises. 

3 What the research doesn't do 

1/ This is not a tutor for precisely constrained tasks with clear cut goals like traditional 
four-part harmonisation or sixteenth century counterpoint. Tutors for this sort of task 
can employ traditional ICAI expert-based or bug-based approaches for tutoring a closed 
set of rules for a precisely defined goal. Examples of intelligeni tutors for such tasks 
include VIVACE (actually an expert component for such a tutor) (Thomas, 85) and 
LASSO (Newcomb, 85). 

21 This research is not designed as an assistant or environment e.g. (Lerdahl and 
Potard, 83) and (Pope, 86) for practised composers who already have well-developed 
compositional skills and strategies. They need tools but not guidance on how to 
compose . 

3/ The tutor cannot be expected to be able to compose good music, or teach detailed 
stylistics, but can be expected to be able to compose music tolerable in small doses, and 
reproduce styles roughly but not closely. To reproduce a style closely currently requires 
enormous effort and computational resources, two of the most notable efforts being 
Ebcioglu (1986) on Bach and Fry (1984) on John Colti&ne. Local features of style can 
be reproduced surprisingly accurately using constraint propagation techniques for 
music (Levitt (85) on Moziui and Fats Waller ). Good first-order approximations of 
styles can be represented very economically using stochastic methods in structured 
frameworks e.g. (Jones, 86) . 

4/ Can the tutor tell the difference between good and bad music? No. But in certain 
circumstances to a certain limited extent (especially if a piece has been assembled u.sing 
devices and materials from existing frames, or is amenable to simple analysis 
techniques (e.g. Winograd 68) it can detect the presence or absence of strategic, 
technical, stylistic and material features. 



4 What's in it for ICAI? 

1/ An amateur composer is by definition one who compo.ses for love. In order to be 
effective, a tutor for amateur composers must take into account in depth not only the 
amateur's skills and errors , but personal likes, dislikes and interests. The sy.stem 
proposed is not .he only ICAI system in which a user's preferences play a key role (see 
(Rich, 79) and (Goldstein, 82) but it is unusual in that preferences play a key role in 
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parallel with the more traditional skills, bugs and core curriculum when dynamically 
constructing teaching plans. 

2/ It is unusual for an ICAI system to try to tackle a domain with the wide range, loose 
defmition and malleability of goals found in music composition. The goal of a 
composition can vary from composer to composer, occasion to occasion and can be 
very hard to state precisely; perhaps "Compose an interesting piece of music" is one of 
the few generally stateable goals (see (Levitt, 81). In practice, the goal for a particular 
piece of music must be more precisely defined eithei by natural limitation (e.g. of 
technique or style) or by deliberate limitation of resources (a frequently used technique 
in the arts). 

3/ This is an ICAI systems making heavy use of a non-text, non- visual communication 
medium (sound and music). 

Implementation plans 

Implementation begins June 87, and a year is available to implement. The system is 
ambitious and is only expected to be partly implemented within that time. The essential 
priorities are to implement 2D harmony space, the rhythm machine, a nucleus of 
example models of musical ideas in Logo (initially without the iconic interface), a 
useable nucleus of domain knowledge frames, and at least one teaching strategy. An 
early prototype of harmony space was implemented in Common Lisp using the 
interface design tool Dialog© on an Apollo Domain workstation controlling a Yamaha 
TX816 synthesizer via a Hinton MIDIC RS232 to MIDI converter in Jan 1987. It is 
intended to make further rapid prototypes of all components and try them out with 
novices as early as possible. 

Problems 

1/ The crudity of existing attempts to formalise mu.sical "knowledge" (despite some 
spectacular and honourable successes). 

21 The vast size of the "space" of music composition, even when limited to a small 
number of areas of music. It is not yet clear to what extent a manageably small number 
of domain models, concepts and heuristics with sufficient generality will have 
reasonable competence for a range of tasks. 
3/ The difficulty of keeping track of what the user is learning. 
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